Web Usage Mining Framework

We present a tool that alleviates the tedious work of data pre-processing and data structuring to support scientific research on web usage mining (WUM). Furthermore, the framework contains web usage mining models as well as methods for basic statistics on the structured data. The contribution of the tool is a tree-like visualization of frequent navigational sequences. This visualization allows for an easily interpretable tree-like view of patterns with highlighted relevant information. The demonstration gives an overview of the functionalities of the framework. 1 WUM framework Web servers are responsible for providing the available web content on user requests. They collect all the information on request activities into so-called log files. Log data are a rich source for web usage mining. Many scientific researches aim at the field of web usage mining and especially at user behaviour exploration. Besides, there is a great demand in the business sector for personalized, customdesigned systems that conform highly to the requirements of users. There are many commercial web access log mining tools in the market. However none of them supports scientific experimentation. We present a framework, developed within the DIANA project [1], to facilitate research on web usage mining. The framework can be used to pre-process and structure web logs and also as a test bed for web usage mining models. Figure 1 shows an overal scheme of the framework. It consists of three main parts: data preparation, session identification Figure 1: An overall scheme of the web usage mining framework and profile mining. The framework handles three sources of information: the web access log data, a mapping table to map visited pages into unique numbers and extra information about the visit and about the users. The first step is to pre-process the data and to load them into a database. In the session identification part we have the user sessions extracted from the database. We can group these sessions based on the extra information. The sessions can either be used in a separate environment as the basis of a research or they can be mined, as the third step indicates in Figure 1, by the built-in algorithms. The framework contains web usage mining models (frequent itemset and association rule mining [2], frequent sequence mining [3], etc.) as well as methods for basic statistics on the structured data. The contribution of this framework is a visualization of frequent navigational sequences. 2 Visualization of frequent navigational sequences The visualization presents a tree-like view of navigational patterns that highlights relevant information and can be interpreted easily. Figure 2 presents an example of such a tree. The tree consists of nodes with their content type related labels in specific colours. There is a special virtual node--the root of all sequences. It contains additional information (name of the tree, support rate). Nodes are connected with lines (edges) in different thickness marking the frequencies of the given paths. Edges are labelled with the percentage of (sub)sequences crossing the given node. The absolute number of sequences finished at the given node is given in parentheses. The size of the tree is controlled by the so-called support threshold. The tree contains only (sub)sessions that are more frequent than this given threshold. Figure 2: An example of tree-like visualization 3 Technology and System requirements • The framework is implemented in Java and therefore it is platform independent. • Additionally, it needs a database management system to store and manipulate data. Any system that supports JDBC like MySQL, Oracle, Sybase, etc., is sufficient.